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ABSTRACT 

Concatenated coding systems utilizing a convolutional code as the inner 
code and a Reed-Solomon code as the outer code are considered. In order to 
obtain very reliable communications over a very noisy channel with relatively 
small coding complexity, it is proposed to concatenate a byte-oriented unit- 
memory convolutional code with an RS outer code whose symbol size is one byte. 
It is further proposed to utilize a real-time minimal-bvte-error probability 
decoding algorithm, together with feedback from the outer decoder, in the 
decoder for the inner convolutional code. The performance of the proposed 
concatenated coding system is studied, and the improvement over conventional 
concatenated systems due to each additional feature is isolated. 
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I. INTRODUCTION 

The complexity of conventional coding systems grows exponentially with the 
block length of block codes (or with the constraint length of convolutional 
codes). To circumvent the prohibitive complexity of directly using very 
long codes, the idea of cascading two or more codes of less complexity to 
achieve highly reliable communications was considered first by Elias [1], 
and later by Forney [2]. Forney's technique of using two or more block codes 
over different alphabets to obLaiu a very low error rate over noisy channels 
is known as concatenated coding. 

Guided by the premise that a convolutional code generally performs 
better than a block code of the same complexity. Falconer [3], and later 
Jelinek and Cocke [4], considered cascading an outer block code with an 
inner convolutional code. Figure 1 shows a general representation of such a 
block-convolutional concatenated coding system. In both the Falconer and 
Jelinek-Cocke schemes, sequential decoding was used for the inner decoder; 
the outer block coding system was used only to intervene when the sequential 
decoder experienced computational overflow. Therefore, these systems can 
be regarded, more or less, as primarily sequentially-decoded convolutional 
coding systems. 

Maximum likelihood (i.e., Viterbi [ 5 ]) decoding of convolutional 

codes with a moderate constraint length can provide an error rate of less 

_2 

than 10 at a rate slightly higher than R co of the noisy channel. 

Forney's work [2] suggested that a concatenated coding system with a power- 
ful outer code can perform reasonably well when its inner decoder is operated 

-2 -3 

with a probability of error in the range between 10 and 10 . It was 
natural then for Odenwalder [6] to choose a Viterbi decoder for the inner 
coding system in his block-convolutional concatenated coding system. 
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Because the output error patterns of Viterbi-type decoders for con- 
volutional codes are bursty, block codes over a large alphabet, such that 
many bits of the inner code form one symbol of the outer code, appear very 
attractive for the outer coding system. The Reed-Solomon (RS; block codes 
are particularly appealing because they can be decoded by relatively simple 
procedures (such as the Berlekamp-Massey [7], [8] algorithm) and have 
optimum distance properties. Because the lengths of the bursts of output 
errors made by Viterbi decoders are widely distributed, it is generally 
necessary to interleave the inner convolutional code so that errors in the 
individual RS-symbols of one block are independent; otherwise, a very long 
block code would be required to operate the system efficiently. Because 
the most likely length of the output error patterns made by the inner decoder 
are on the order of the constraint length, K, of the convolutional code, 
Odenwalder chose the RS symbol alphabet to be GF(2 ). 

In a block-convolutional concatenated coding system such as Odenwalder' s 
employing a Viterbi decoder with conventional convolutional codes, it is 
very unlikely that the beginning of a decoding error burst is always aligned 
with the boundary between two RS symbols; in fact, such a burst only two 
bits long may affect two RS symbols. This fact led us to consider using 
good convolutional codes which are symbol-oriented rather than bit-oriented. 
In [9], we reported a class of unit-memory convolutional codes for which 

k -bit information segments are encoded into n -bit encoded segments. It 
o o 

• was shown there that an (n , k ) convolutional cede with unit memory always 

o o 

achieves the largest free distance possible for codes of the same rate 

Mk 

k / n and the same number 2 o of encoder states, where M is the encoder 
o o 

memory. The unit-memory codes are naturally byte-oriented with byte size 

equal to k information bits. It will be shown that the improved free 
o 


distance and the symbol-oriented nature of these codes provides an improve- 
ment of approximately 0.3db in the overall performance of the 
concatenated coding system when these codes replace bit-oriented convolutional 
codes. 

Another improvement is to modify the decoder for the convolutional code 
so that the decoder emits not only the most-likely estimated symbol, but 
also reliability information about the estimated symbol. The outer decoder 
may then use this reliability information to perform either "erasures-and- 
errors" decoding or "generalized-minimum-distance" (GMD) decoding as 
suggested by Forney [2]. Zeoli [10] and Jelinek [11] proposed to extract 
reliability information by annexing a long tail to the original convolutional 
code and using this added tail to provide an error detection capability for 
the estimate made by the Viterbi decoder for the original shorter convolutional 
code. This approach requires the feedback of symbols previously encoded by 
the Viterbi decoder and, more importan .ly, vses the output of the outer 
decoder to restart the inner Viterbi decoder whenever an error is corrected 
by the outer devoder. It will be shown that the error detecting capability 
used with an "erasures-and-errors" outer decoder provides an improvement of 
0.2 db and that the feedback from the outer decoder further improves the 
performance by 0.3 db. 

An alternative approach to extracting reliability information from the 
inner decoder is to compute the a posteriori probability of correctness for 
each decoded symbol from the decoder for the short constraint length con- 
volutional code and then use this probability as the reliability information 
provided to the outer coding system. It will be- shown that, when used 
with an errors-and-erasures outer decoder, this scheme improves performance 
by only 0.05 dB to 0.1 dB compared to hard-decision decoding and hence is 
less powerful than Zeoli's tail annexation scheme; yet its performance is 


undoubtedly optimal among all schemes employing only the short constraint 
length convolutional code (with no annexed tail). However, it will be shown 
that, in conjection with the use of feedback from Che outer decoder, the 
a posteriori probability inner decoder provides about 0.2 db more improve- 
ment than does the Viterbi decoder aided by feedback. In fact, the a posteriori 
inner decoder, used with feedback from the outer decoder, offers a slight 
improvement over Zeoli's scheme; moreover the inner encoder and the inner 
decoder have the same constraint length so that the inner decoder generally 
and automatically returns to normal operation only a few branches after 
making an error. 

The plan of this paper is as follows. In Section II, a "real time" de- 
coding algorithm for unit-memory convolutional codes is developed which 
calculates the a posteriori probability for each value of the byte being de- 
coded. In Sections III, IV, and V, the performances of several b'lock- 
convolutional concatenated coding systems having unit-memory convolutional 
inner codes are compared with similar systems having conventional bit- 
oriented convolutional inner codes. In each case, we chose the (18,6) unit- 
memory convolutional code as the inner code because it has practically 
minimum complexity in terms of decoder implementation, and because of its 
reasonably large free distance (^f ree = 16). We chose the Reed-Solomon codes 
over GF(2^), with block length 63 symbols, as the outer codes so that the 
symbol size of the RS codes would be matched to the byte-size (six bits) of 
the unit-memory code. In Section VI, the degradation of performance, when 
the rate 1/3 inner convolutional code is replaced by a rate 1/2 convolutional 
code, is considered in rrder to demonstrate the tradeoff between bandwidth 
expansion and signal-energy-to-noise ratio. In Section VII, the 95% con- 
fidence intervals for the simulation results are obtained and interpreted. 


I 

l 


I 

l 


II. REAL-TIME MINIMAL-BYTE-ERROR PROBABILITY 
DECODING OF UNIT-MEMORY CODES 

We now develop en algorithm for real-time minimal-byte-error probability 

decoding of the unit-memory convolutional codes described in [9]. 

Let a (t = 1,2,...) denote the byte (or subblock) of k information 
— t o 

bits to be encoded at time t, and let _b t (t = 1,2,...) be the corresponding 


encoded subblock of n^ bits. For a unit-memory code, 

^ - St G 0 + £,_! C 1 


( 1 ) 


where G_ and G, are k x n matrices and where, by way of convention, 3^=0. 
We assume that the sequence b^, b^* ... has been transmitted over a discrete 
memoryless channel and that _r (t = 1,2,...) is the received subblock corre- 
sponding to the transmitte'’ subblock _b . We shall write a, to denote 

[a^., a.. +] , ... a_ t t ] ; similarly for b^ j and r^ t , j . 

By real-time decoding with delay A, we mean that the decoding decision 

for a is made from the observation of r M ^ . The real-time minimal- 

-t -[1, t + A] 

byte-error probability (RTMBEP) decoding rule then is that which chooses 

its estimate a as that value of a which maximizes P(a |r r , , , -i ) for 
— t — t — t — 11, t + AJ 

t = 1,2 To find a recursive algorithm for this decoding rule, we 

begin by noting that 

na t l iUit+i) )=PU t ,r Uit+4] )/IP( 2 , r(lit+il ) (2, 

where we have used a to denote a running variable for a . It suffices then 

— — t 

to find a recursive method for calculating P(£ , _r , ,). 

We next observe that 

P(i t li [l.t + 4] ) ' P( V i[l,t] )1>( i[t + l, t+il'V ifl.t] 5 


‘ P( V £ll.t] )P( -It+l. tWV 


(3) 


X 


where we have used the facts that the channel is memorylesu and that the 

code has unit memory. It remains to find recursive rules for obtaining 

the two probabilities on the right in (3). 

Obtaining the recursion for P(a , r M .) is quite standard [12]- [14]; 

“t — 1 1, t J 

P( -V i[i,t] ) = J p( -[t-i, t]» -[I,t] } 


Vi 


I L [lt t-1] )p( ^t • It^t-l* — [1, t-l] ) ‘ 


(4) 


Vl 


But also 

•<v -Etk-r iu, t-i]> - tr iu, t-i]> 

- 2 ° p (E t lt(a It . 1( t] » (5) 

where we have written b(a, . , ) for the value of b determined by (1) 

l 1 f t J t 

from a, and where we assume here and hereafter that all information 

1 1— 1 , t J 

sequences are equally likely (as corresponds to maximum-likelihood decoding.) 
Substituting (5) into (4), we have our desired recursion 

P( V - 2 ° I p <St-i* *[i, t-i] )p( itl ^u-i, tj”- (6) 

a 


*t-l 


We now turn to the quantity P(£j t+1 t+A ]liL t ) which we note is the 
i *= 1 value of 

p (£f^ -J.A1 = I P ^— t+i’ — [t+i, t+Aj^+i-l*' 


v -[t+i, t+A] 


(7) 


*t+i 


Proceeding in the same manner that (6) was obtained from (4), we find the 
desired recursion 

p( i[ t+ i, t+M'-t+i-i 5 ■ 2 °£ p( i[t + i+i,t+i) li t+i )p<I t+i | - ( -[t+i-i.t + i] )) - (8) 

-t+i 


This recursion is initialized with Its i = A value 



I 


P ^t+J-t+A-l? “ E P( -^t+A* 


■^t+A 


-k 


- 2 ° l P ^4.,lt( 


•^t+A 


■^t+A 1 — v3 [t+A-l, t+A] )) ‘ 


(9) 


and evaluated with i = A-l, A-2, . 1. It should be noted that, because 
of the restriction to unit-memory codes, the recursion (8) is much simpler 
than the corresponding one required for RTMBEP decoding of general con- 
volutional codes [14]. 

An algorithm to carry out the recursive rules given by (6) and (8) 
requires, for each byte (or "state" in the usual Viterbi decoding terminology) 
c^, the storage of two real numbers, f(a) and h(a); namely, 

f(a) = P(£ t - a, t j) (10) 


and 


*>(£) - p <£ [t+i , t+ai'^t+i-i ■ 2) 


(ID 


where i will be decremented from A to 1 as the algorithm progresses. 

(Of course, the received segment t+A] must a l so st° re d so that 

P(r ,.|b(a, . ,,.)) can also be found for i = A, A-l, ..., 1.) We may 

— t+i 1 lu+i-1, t+ij 

now state: 

The RTMBEP Decoding Algorithm for Unit-Memory Codes 

k 

Step 0 : Set f ( 0) = 2 ° and set f(a) = 0 for a ^ (). Set t 
Step 1 : Make the replacement, for all states a, 

-k 

f (a) «- 2 ° l f(a’) P(r |b(a', a)). 


1 . 


Set i = A and, for all states ja, set 
-k 

h(a) ■ 2 ° [ P(r t+/l |b(a, a’)). 


Step 2 : 


Step 3 : Decrease i by 1 and make the replacement, for all states a, 

-k 

h(a) 2 0 l h(a')P(r .. |b(a, a')), 

a' _tl 

If now 1 ** 1, go to Step A. Otherwise, return to Step 3. 

Step A: Emit, as the estimate of a^ , that byte a which maximizes f(a)h(a), 

— t — o — — 

and emit, as the reliability indicator, the probability 

P( it “ Sj-U. t+4] ) ' f( V h( 2 D )/ l f <S )h <S>- 

Increase t Dy 1 and return to Step 1. 

The only feature of the algorithm that should require any comment is 

k Q 

the initialization of f (0) at 2 . This is required so that the first time 

step 1 is performed one obtains the correct initial value f (a) =• P(r^|b/(), a)). 

In fact, however, it makes no difference in the output from the algorithm 

if the f and h values are scaled by fixed positive constants, oO lhat 

-k Q 

f(0) = 1 is permissible in Step 0 and the factors 2 can be removed in 
Steps 1, 2 and 3. 

Note that Step 3 of the algorithm, which has the same complexity as 

Step 1, is performed A - 1 times for each time that Step 1 is performed. 

It is clearly desirable then to keep A as small as possible. Table I shows 

the variation of the decoding byte-error probability, P_„, with the decoding 

Bb 

delay, A, for the (n = 18, k = 6) unit-memory code of [9] used on a 

o o 

simulated three-bit- quantized additive white Gaussian noise (AWGN) channel. 

We see that A = 8 gives virtually the same P as the "optimum" choice 

Bb 

A “ 

We now point out, however, that one can reduce the ratio of Step 3 
operations to Step 1 operations to as close to unity as desired without any 
degradation in performance but at the cost of additional storage. The 
"trick" is to use a variable decoding del ay A. Each a is decoded from 


I 


I 


' 

I 
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Table I. Variation of Decoding Byte-Error Probability p with Decoding 

Delay A for RTMBEf 1 Decoding of the (18,6) Unit-Memory Code on 

a Simulated AWGN Channel with an E’/N of 1.25 db. 

b o 

(4000 bytes decoded for each A). 


A (bytes) 

4 

6 

8 

16 

P 

.0285 

.0248 

.0193 

l 

.0193 

1 


Table II. Byte-Error Probability, p, for Viterbi Decoding of Three 

R * k /n = 1/3 Convolutional Codes on a Simulated AWGN 
o o 

Channel. (8000 bytes decoded for each point shown, decod- 
ing delay A in bits of 48 in all cases.) 


E^/N 


O 


p (95% confidence) 
(18,6) unit-memory 

p(95% confidence) 

M = 6, (3,1) code 

p(95% confidence) 

M *= 7, (3,1) code 



1.00 db 

1.25 db 

1.50 db 

1.75 db 

code 

.0305 (+.0053) 

.0200 (+.0044) 

.0118 (+.0033) 

1 

.0065 (+.0025) 


.0488 (+.0068) 

.0325 (+.0056) 

.0233 (+.0048) 

.0128 (+.0035) 


.0400 (+.0062) 

.0225 (+.0047) 

.0140 (+.0037) 

.0103 (+.0032) 



Figure 1. 


A concatenated coding system employing a convolutional 
code as the inner code and a block code as the outer code. 
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P(a t |r^ t+A]^ ^ Ut depending on c ^ e value of t, takes some value in the 

range A < A < A„. The minimum decoding del *r f a v is chosen large enough 

to ensure negligible degradation, say A^ ■ 8, while the maximum decoding 

delay, A u . is chosen small enough to make the increased memory tolerable 
M 

as will soon become apparent. 

In this variable real-time minimal-byte-error probability (VRTMBEP) 

decoding, one stores A,. - A + 2 real numbers for each state a, namely: 

f . (a) for i * 1,2,..., A u - A +1 and h(a) where 
1 — Mm - 


f i<*> * - -a- r u , t+i-!]) 


(12) 


and where h(oi) is as in (11) with A replaced by A^. 

Observe now that, in the process of executing Step 2 of the RTMBEP 


algorithm with A = A^, one would obtain sequentially the quantities 


P(r 


a) 


'-[t+i, t+A M ]|--t+i-l 


(13) 


for i = A -1, A -2, ..., 1. But the product of the quantity in (13) with 
M M 

f ^ (o) as in (12) is, according to (3), equal to P(a. t+i _^ = £lL{^ j)i 

this is precisely the statistic needed to estimate with a decoding 

delay of A = A^ - i + 1. Hence, if we had the foresight to perform Step 1 
of the RTMBEP algorithm A^ - A^ + 1 times and to store the resulting f ^ ( ot ) , 
then we could make A^ - A^ + 1 decoding decisions during the A M - 1 times 


M 


that Step 3 is performed. Thus, for each time we use Step 1, we would be 

using Step 3 only (A -1)/(A U - A +1) times. For instance, with A = 8 and 

M Mm m 

A = 13, we would perform Step 3 only twice for each time we performed Step 

M 

1; and we would be storing only A - A +2=7 real numbers per state rather 

N m 

than 2 as in the original RTMBEP algorithm in which Step 3 is performed 
A - 1 ■ 7 times for each time that Step 1 is performed. 


It should now te obvious that the following algorithm is the necessary 
modification to the .RTMBEP decoding algorithm for obtaining reduced computation 


at the price of additional storage as has Just been described. 


The VRTMBKP D e coding A 1 gorit hm for Unit-Memory Codes 


Step 0 ; Set f 


- A 


"iH 1 

Set t ■ 1. 

-k 


(0) 


and set f 


- V 1 


(c0 = 0 Li u | 0. 


Step 1 : Set f-(a) = 2 ° T f 


a' M 
-k 


A.. - A +1 
m 


(a' )P(£ t |b(a' , a)) , 


and set f i+1 (a) =2 ° ^ f ^ (o)P(£ t+i |b («' , u)) 


for i = 1,2,..., \„ - A in order. 

M m 


Step 1: Set i = A and, .'or all states o_, set 

h(ot) 


-k 

2 I p (£ t+ , |b<H. i')). 

a' >1 


Step 3 ; Decrease i by 1 and ma^c the replacement, for all stales a, 

" k ° r 

h (a) *2 \ h(a' )P(r . |b(a, ex' )). 

a' ‘• +1 

If now i _< - A^ + 1, go to Step A. Otherwise, return to Step 3. 

Step A: limit, as the estimate of a that byte cx which maximizes 

' — t+i-1 1 — o 

fj(j0h(ni), and emit, as the reliability indicator, the probability 

PCo. . . i ’ “ l r M «, •)) = f.(a )h(a )/ l f (a)h(a) . 

— t+i-1 — o — L i • t+A,|J i — o — o L — — 

M a 

If i = 1, inc r ease t bv /»„ - A + 1 and return to Step 1. 

M m 

Otherwise, decrease i by 1 a.id return to Step 3. 

It is satisfying to note that the VRTMllEP decoding algorithm reduces 

to the RTM11EP algorithm when A., 11 A . It sliculd be pointed out that when 

M m 

only a finite number, L, of information bytes are encoded and one takes 


A.. “ L, the largest possible choice, then the VRTMBFP algorithm reduces to 

M 



I 
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that iven by Bahl et^ al_. (12] (when the latter algorithm is specialized to 
unit-memory codes) and does about twice the computation of the usual Viterbi 
decoder; but this case also maximizes the memory requirements. The chief 
advantage which both RTMBEP and VRTMBEP decoding of unit-memory codes have 
over Viterbi decoding is in their providing reliability information about 
the decoding decisions; information of considerable value to the outer de- 
coder in a concatenated coding system. 

Because the resulting performance of the RTMBEP and VRTMBEP algorithms 

* 

are indistinguishable when A = A^ is chosen large enough for negligible 
degradation compared to A = °°, say A^ = 8, we will not hereafter distinguish 
between the two algorithms in our discussion of concatenated coding systems. 

III. ODENWALDER'S CONCATENATED CODING SYSTEM AND SOFT-DECISION 
MODIFICATION WITH THE RTMBEP DECODING ALGORITHM 


The concatenated coding system proposed by Odenwalder [6] , which we 
shall call System I, is as shown in Figure 1 where the inner decoder is a 
hard-decision Viterbi decoder and where the outer decoder is a t-error 
correcting decoder for the RS outer block code. Here and hereafter, we 
assume that the interleaving is "perfect", i.e. , that the symbols in each 
RS block at the output of the interleaver have been independently decoded 
by the inner Viterbi decoder. Thus, we can then calculate the probability 
of a decoding error in an RS block, P^^ , as 


I <”> rV-p)"- 1 , 


n-i 


ERS 


(U) 


i=t+l 


where n is the RS block length (in bytes) and p is the byte-error probability 

at the Viterbi decoder output. Further, since almost all the incorrectly 

decoded RS codewords are d . = 2t + 1 symbols away from the correct code- 

min 

. is the minimum distance of the R5 code), the byte-error 
min 


word (where d 
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probability, P UK , of the concatenated coding system is given closely by 

2t+l „ 


BE 

P 


BE n 


ERS' 


(15) 


For a byte size of 6 bits, as will be assumed hereafter, the RS code 

nas length n = 2^ - 1 = 63 bytes. For convenient reference, we give in 

Table II the byte-error probability of a Viterbi decoder for the three 

different convolutional codes of rate R„«.. = k /n = 1/3 that will be used 

CON o c 

in our subsequent comparisons when used on four different AWGN channels; 
this data is taken from [9]. The AWGN channels are specified by the ratio 
of channel energy per encoder input bit to one-side noise power spectral 
density, E^/N^. Note that the energy per channel input bit (decoder output 

bit), E_ s given by E g = R C0 N E b* But a ^ so = ^RS E b w ^ ere ^rs is t * ie 
rate of the RS code and is the channel energy per information bit entering 
the RS encoder. Thus, the channel energy per information bit to one-sided 
noise power spectral density ratio for the overall concatenated coding system, 
E^/N q is given by 


E l /N = 


(E /N ). 


b ° P TtS R CON ’ S °' 


(16) 


Using the results of Table II together with (14) and (15) , we can 
calculate the byte-error probability for Odenwalder's System I for various 
RS outer codes. The results of this calculation are shown in Fig. 2 for 
the three different R n = 1/3 convolutional codes, namely (i) the con- 
ventional (3,1) code with H - 6, i.e., K = 7; (ii) the conventional (3,1) 
code with M = 7, i.e., K = 8; and (iii) the (18,6) unit-memory code. Codes 
(i) , (ii) and (iii) have free distances of 15, 16 and 16, respectively, and 
their corresponding Viterbi decoders have 64, 128 and 64 states, respectively. 
We see, from Fig. 2, tin*' 'C use of the unit-memory code provides an advantage 
of about 0.3 db over the conventional code with the same state complexity, 
part of which gain is attributable to the larger free distance of the unit- 


Concatenated with: 
O t = 4 RS code 
a t = 6 RS code 
0 t = 8 RS code 


Sys-tem I with 
M = 6, (3,1) code 


System I with 
M = 7 , (3,1) code 


System I with 

(18,6) unit-memory code 


System II with 

(18,6) unit-memory code 


l I _l J I J I L 

1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 


E./N (db) 
b 0 


Figure 2. The performance of Concatenated Coding Systems I and 
II with RS codes over GF(2k) on a simulated AWGN 
channel with E' /N = 1.25 db. 



X 
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memory code. But the unit-memory code is also about 0.1 db superior to the 
conventional code with the same free distance (and doubled number of decoder 
states); this gain is attributable entirely to the byte-oriented structure 
of the unit-memory code. 

It should be mentioned that gains of 0.1 db are not insignificant in 

concatenated coding systems. As can be seen from Fig. 3, a gain of 0.1 db 

corresponds co a reduction of P by nearly an order of magnitude, such 

steepness of the P vs. E, /N curves being characteristic of well-designed 
BE b o 

concatenated coding systems. 

The inner decoder, i.e., the Viterbi decoder, in System I makes "hard 
decisions" on the decoded bytes. The system performance can be improved by 
using a "soft decision" decoder which passes along to the outer decoder a 
reliability indicator for each decoded byte. Such a system, in which the 
inner decoder is a RTMBEP decoder and the outer decoder is an errors-and- 
erasures decoder for the RS code, will be called System II . (For ease of 
reference, we summarize in Table III the characteristics of each of the six 
concatenated coding systems that will be considered in this paper.) When 
the reliability indicator, PO^Jr.^ t+A]^ ^ or a decoded byte is less than 
some specified T, the outer decoder treats the byte as having been "erased." 
The erasures-and-errors decoder for the RS code can correct t errors and e 

erasures, whenever 2t + e < d . . Thus, the block error probability for 
’ min 


the outer decoder is given by 

n d/2 


P FRS - I l < t V pVu-p-q)"-” <1 7 > 

d=d . t=0 t,e 
mm _ 
e-d-2t 

where p is again the byte-error probability for the inner decoder, where q 
is the byte-er-sure probability for the inner decoder, and where 

i 


, n-t-e 


( n s „ nJ . 

t ,e. tie! (n-t-e) ! 


1 
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Table III. The Six Bloc. -Convolutional Concatenated Coding Systems Studied. 

(EO = errors only decoder, E + E = errors and erasures decoder, 
FBTID = feedback to inner decoder.) 



Inner Decoder Type 

Outer Decoder Type 

Inner Code Tail Annexation 

System I 

Viterbi hard-decision 

EO 

NO 

System II 

RTMBEP soft-decision 

E + E 

NO 

System III 

Viterbi hard-decision 

EO with FBTID 

NO 

System IV 

RTMBEP hard-decision 

EO with FBTID 

NO 

System V 

RTMBEP soft-decision 

E + E with FBTID 

NO 

System VI 

Viterbi soft-decision 

E + E with FBTID 

YES 


Table IV. 


Variation of Inner Decoder Byte-Error Probability p and Byte- 
Erasure Probability q and of Outer Decoder Byte-Error Probability 
P with the Erasure Threshold T for the (18,6) Unit-Memory Code 
on a Simulated AWGN Channel and with the Minimum Distance d . of 


the Outer RS Code. 


min 


V N o 

in db 

T 

P 

1 

q 

P BE for 

d . =9 

min 

P BE f0r 

d . « 13 

min 

P BE £ ° r 

d . = 17 

mm 

1.00 

.70 

.01325 

.04150 

.740xl0" 2 

. 477xlO~ 3 | 

.128xlO _A 

II 

.50 

.02100 

.01950 

. 677xlO~ 2 

. 555xl0~ 3 

. 213x10~ A 

II 

.80 

.00675 

.03400 

. 123xlO~ 2 

.244x10 _A 

. 193xl0~ 6 

1.25 

.70 

.00800 

.02650 

. 902xl0~ 3 

. 179x10~ A 

.149x]0 _6 

II 

.50 

.01350 

.01125 

.107xKf 2 

.332x10 

-6 

.481x10 

II 

o 

qo 

• 

. 00425 

.02125 

. 112xlO~ 3 

.69lxl0“ 6 

. 173x] 0 -8 

1.50 

.70 

.00525 

.01625 

.981xl0 -A 

,684xl0 -6 

. 204xl0 _8 

II 

.50 

.00900 

.00400 

. 113xlO~ 3 

. 136xl0 -5 

. 774xl0~ 8 

II 

o 

oo 

.00250 

.01050 

. 416xl0~ 5 

. 636xl0~ 8 

.416xl0 _11 

1.75 

.70 ; 

.00250 

.00825 

. 249xl0~ 5 

. 334xlO _8 

.196xl0 -11 

II 

.50 1 

.00400 

.00250 

. 336xl0~ 5 

. 816xl0~ 8 

.927xl0 -11 
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The byte-error probability of the overall system is again obtained from (15). 
The byte-error-probability, p, and the erasure probability, q, depend 


on the particular threshold, T, specified. The optimal threshold is a 

function of E’/N and the minimum distance, d . , of the Reed-Solomon code, 
bo min’ 

Roughly speaking, for a given block length n, as d^ gets larger, the over- 
all block error probability is minimized at a higher erasure rate. Ue have 
found no simple way to determine the optimal threshold analytically. Instead, 
we have found p and q for T = 0.5, 0.7 and 0.8 by simulation and have used 
these values of p and q to calculate the byte-error probability of the coding 
system. In Table IV, we show the result of this calculation. We see, for 


E^/N q in the range from 1.25 db to 1.75 db, that T = 0.7 is the best threshold 


among the three candidates. 


The performance of System II with T = 0.7 is also plotted in Figure 2. 


The improvement over System I of the performance due to the erasure scheme, 
as observed from Figure 2, is dependent on the error correcting capability 
of the outer coding system as well as on E^/N^ and is approximately 0.1 dB. 
This slight improvement is probably not significant enough to justify the 
increased complexity of the RTMBEP decoder over the Viterbi decoder. However, 
as we shall soon see, the RTMBEP decoder coupled with an "erasures-and-error" 
block decoder performs much better than the Viterbi decoder when feedbac k 
from the outer decoder is utilized. 


IV. FEEDBACK FROM THE OUTER DECODER TO THE INNER DECODER 


Because of the nature of a convolutional code and the Viterbi decoding 
algorithm, once an "error event" occurs the decoder often makes a number of 
closely spaced erroneous estimations before it recovers to correct operation. 
Since the outer decoder of a concatenated coding system is designed in such 
a way that it is able to detect and correct almost all of the errors made 
by the inner decoder, it is then of significant advantage if the corrected 
estimates of the outer decoder are fed back tt> restart the inner decoder 
from the point where it first erred in order to eliminate the "burst" of 
errors. Figure 3 illustrates the general concept of such a block-convolutional 
concatenated coding system. 

To study the gain provided by feedback from the outer decoder, we first 
implemented a software Viterbi decoder and a software RTMBEP decoder which 
can be restarted with feedback. Assuming that the outer decoder always makes 
correct decisions, a justifiable assumption since the probability of byte- 
error at the outer decoder output- is at least several orders of magnitude 
less than that at the inner decoder's output, we obtained the results shown 
in Table V for the (18,6) unit-memory convolutional code on a simulated AVON 
channel with an E'/N of 1.25 db. From Table V, we see that the RTMBEP 
decoder receives a considerably greater benefit from the feedback than does 
the Viterbi decoder. We then considered the following block-convolutional 
concatenated coding systems: 

S ystem III: A hard-decision Viterbi inner decoder with feedback from 

the errors-only RS outer decoder, i.e., System I with feedback. 

System IV : A hard-decision RTMBEP inner decoder with feedback from the 
errors-only RS outer decoder. 
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Figure 3. A block/convolutional concatenated coding system with 
feedback from the outer decoder to the inner decoder. 


Table V. The Effect of Feedback from the Outer Decoder on the Byte-Error 

Probability for a Viterbi Decoder and an RTMBEP Decoder on a 

Simulated AWGN Channel with an E'/N of 1.25 db. (8000 bytes 

bo 

decoded for each point shown, decoding delay A of 48 bits in 
each case.) 



p for (18,6) unit-memory code 
(95% confidence) 

p for M = 7, (3,1) code 
(95% confidence) 


No feedback 

With feedback 

No feedback 

1 With Feedback 

Viterbi Decoder 

.0200 (+.0032) 

.0110 (+.0023) 

.0225 (+.0034) 

.0133 (+.0025) 

RTMBEP Decoder 

.0193 (+.0031) 

.0075 (+.0019) 
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System V : A soft-decision RTMBEP inner decoder with feedback from the 

erasures-and-errors RS outer decoder, i.e. , System II with feedback. 

The performances of Systems III, IV and V when used with the (18,6) 
unit-memory code on the AWGN channel are shown in Fig. 4. For ease of 
comparison, the corresponding performances of Systems I and II, given in 
Fig. 2, are repeated in Fig. 4. By comparing performances between Systems 
I and III, we see from Fig. 4 that feedback from the outer decoder improves 
the system by about 0.3 db for a hard-decision Viterbi inner decoder. As 
can be seen from Table V, the performance of a hard-decision RTMBEP inner 
decoder is virtually indistinguishable from that of a Viterbi inner decoder 
for a unit-memory code; thus, the performance of System I in Fig. 4 is also 
the performance of the system with an RTMBEP inner decoder without feedback 
from an errors-cnly RS outer decoder. Hence, by comparing the performances 
of Systems I and IV in Fig. 4, we can conclude that feedback from the outer 
decoder improves the system by a full 0.5 db for a hard-decision RTMBEP inner 
decoder. By comparing the performances of Systems IV and V in Fig. 4, we 
can further conclude that, when feedback from the outer decoder is used, an 
additional 0.1 db improvement can be gained by using a soft-decision RTMBEP 
inner decoder rather than a hard-decision one — the same improvement as was 
observed in the previous section when there was no feedback from the outer 
decoder. 

V. ZEOLI'S TAIL ANNEXATION SCHEME APPLIED 
TO A UNIT-MEMORY CONVOLUTIONAL CODE 

In [10], Zeoli proposed a concatenated coding system that employed a 
rather long constraint length (K = 32, i.e., M = 31) convolutional code 
obtained by annexing a long tail to the M - 7, (3,1) convolutional code. 

The longer code is then decoded by the same Viterbi decoder as for the short 
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code with the exception that the Information sequence along the best path 
to each state is treated as correct and used to "cancel" the effect of the 
longer tail from the encoded sequence. Thus, the decoder state complexity 
remains the same as that for the original code and the annexed tail has 
absolutely no effect on the hard-decision decoding error probability until 
after an error has been made . But the tail provides excellent "error-detection" 
once the Viterbi decoder starts to make mistakes. Because the tail is not 
cancelled when a decoding error is made, the state metrics become extremely 
ominous after a few decoded branches and can be used as the basis for excellent 
erasure rules for the output of the inner decoder. However, feedback from 
the outer decoder is no longer an option, but now a necessity in order to 
reset the decoder to the correct state and thus to terminate the very "error 
propagation" used to trigger the erasure alarm. 

To study the improvement resulting from Zeoli's scheme, we annexed, to 
the (18,6) unit-memory convolutional code, a three-branch-long "random tail" 
such that the resultant code is actually an M = 4, (18,6) convolutional code. 

The encoding matrices of this latter convolutional code are shown in Table 
VI. The length of the tail was chosen to be comparable in memory to the 
M * 31, (3,1) code used in [10]. (Because the decoder is intended to make 
mistakes continually after its first error, it makes no difference whether 
the annexed M = 4, (18,6) code is catastrophic [15] or not.) The last of 
the systems to be considered in this paper. System VI , is that of Zeoli [10], 
namely a soft-decision Viterbi inner decoder with feedback from an errors- 
and-erasures RS outer decoder, with the M = 4, (18,6) code replacing his 
conventional M * 31, (3,1) code. 

The state metric used in the "real time Viterbi decoder" [14] of System 
VI, namely >i<t+i) - log P(4 (1> t+i] ll (lf t+4) when a (] _ t+4 , is the "best 
path" at time t+A, can be used as the basis for an effective erasure 
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Table VI. The Encoding Matrices of the M “ 4 (18,6) Convolutional Code 


Obtained by Annexing a Randomly-Chosen Tail to the (18,6) Unit- 


Memory Code. 


*111000 

110100 

110000T 

r 000011 

000111 

001011 

011100 

011010 

011000 


000110 

001110 

010110 

001110 

001101 

001100 

G. - 

001100 

011100 

101100 

000111 

100110 

000110 

1 

011000 

111000 

011001 

100011 

010011 

000011 


110000 

110001 

110010 

110001 

101001 

100001 


100001 

100011 

100101 


*000110 

000001 

101111 ’ 


'011000 

111001 

011000’ 

100011 

000011 

010011 


110001 

110010 

110000 

110001 

100110 

100001 

G, - 

100011 

100101 

100001 

111000 

110101 

001000 

3 

000111 

000011 

001011 

011000 

011010 

011100 


000110 

001110 

010110 

001100 

011100 

110110 


001100 

011100 

101100 


6 


4 


111111 

000111 

000000 

100000 

010100 

111010 

ilk 


010100 

111010 

mill 

000111 

000000 

100000 


000000 ' 
100000 
010100 
liioio ; 
linn | 
000111J 


Table VII. Variation of Inner Decoder Byte-Error Probability p and Byte- 

Erasure Probability q and of Outer Decoder Byte-Error Probability 

P__, with the Erasure Parameter X for the M = 4 (18,6) Code 

Obtained by Annexing a Tail to. the (18,6) Unit-Memory Code on a 

Simulated AWGN Channel with an E^/N q of 1.25 db and with the 

Minimum Distance d . of the Outer RS Code, 
min 





P BE for 

P BE for 

9 BE £ ° r 

X 

p 

q 

d min - 9 

d . =13 

min 

d , = 17 

min 

..50 

.00125 

.03788 

2.095xl0~ 4 

8.175xl0” 7 

9. 465x10” 10 

• 

oo 

o 

.00263 

.02088 

3.602xl0“ 5 

1.074xl0” 7 

1.245xl0~ 10 

:.oo 

.00425 

.01450 

4. 168xl0” 5 

1.899xl0~ 7 

3.708xl0" 10 
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rule as follows. The difference, u(t+A) - p(t), Is, along the correct encoded 
path, the sum of An Q statistically independent random variables, each corre- 
sponding to one encoded bit. Note that, for System VI, An^ - 8(18) - 144. 

The central-limit -theorem can thus be invoked to assert that p(t+A) - p(t) 
is approximately Gaussian. Letting m and o be the (easily calculable) mean 
and standard deviation of u(t+A) - p(t), it is natural to use the erasure 
rule : Erase whenever w(t+A) - u(t) is more than 1 ccandard deviations 

above m. In Table VII , we give the performance of System VI using this 
erasure rule for X ■ 1.5, 1.6 and 2.0; the value 1.8 is seen to give the best 
performance. Note that if y(t+A) - y(t) were truly Gaussian, the probability 
that it would exceed m +* 1.8o (i.e., the probability of an erasure in the 
Viterbi decoder output) would be .036; the observed value- of .021 given in 
Table VII is rough confirmation of the appropriateness of the Gaussian 
approximation. 

The performance of System VI on the AWGN channel is shown in Fig. 5; 
for comparison, the performance of Zeoli's original system, taken from [10], 
is also shown. The performance of Systems III and V, given in Fig. 4, are 
also repeated in Fig. 5 to indicate ho v System VI compares lo r he systems 
previously considered. By comparing the performance c i Systems III and VI, 
we see that Zeoli's tail annexation scheme (and the resulting erasure 
capability) has improved the performance of the feedback system with a Viterbi 
inn - decoder by about 0.2 db. 

VI. DEGRADATION OF PERFORMANCE FOR EMPLOYING 
HIGHER RATE INNER CODES 

We have studied, rather extensively, block-convolutional concatenated 
coding systems employing rate 1/3 convolutional codes and Reed-Solomon codes 
over GF (2^). However, it is sometimes desired in practice to operate the 



System III 
(Unit-memory code) 


System VI (Zeoli) 

(M - 7, (3,1) code) 


System VI 
(Unit-memory code.) 

System V 

(Unit-memory code) 


_j l l i- l 1 J 1 — - 
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*' t ;rf ormance of Zeoli's tail annexation scheme (System 
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and comparison with other concatenated coiiinp, systems. 
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inner convolutional codes at a higher rate (i.e., narrower bandwidth), rate 
1/2 in particular, in order to ease the burden imposed on the phase-lock 
loops in the receiver. We now describe an heuristic approach to estimate 
the performance of similar concatenated coding systems with rate 1/2 coding 
systems from the rate 1/3 results. 

From past experience [16], it has been observed that the performance 
of a rate 1/2 convolutional coding system is about 0.5 db inferior to that 
of a rate 1/3 convolutional coding system of the same complexity. To verify 
the general applicability of this rule-of-thumb, we used a hard-decision 
Viterbi decoder (without feedback) for an M = 6, (2,1) convolutional code 
on a simulated AWGN channel at E^/N^ = 1*75 db, or, equivalently, E g /N o = 

-1.25 db. The results of this simulation and the calculated overall byte- 
error-probability when this decoder is used with an errors only RS outer 
decoder concatenated with Reed-Solomon codes are given in Figure 6. For 
comparison, the performance of the similar R = 1/3 system employing the 
M = 6, (3,1) code is also shown. We see f : < \ Fig. 6 that the latter system 
is about 0.5 db superior to the former. It seems ruasonaole then to conclude 
that a concatenated block-convolutional coding system with a rate 1/2 inner 
code will be about 0.5 db inferior to that with a rate 1/3 inner code for 
the same number of decoder states for the Viterbi inner decoder. 

VII. CONFIDENCE INTERVALS FOR THE SIMULATION RESULTS 

In the preceding, we have reported the performances of numerous block- 

convolutional concatenated coding systems. The overall byte-error rate was 

calculated from the byte-error rate of the inner decoder as obtained by 

simulation. The rather large values of P D „ for the inner decoding imply 

that the simulations require only a modest sample size. Assuming that the 

decoder makes an error with probability P or independently for each byte-decision, 

dL 



Concatenated with: 
t = 4 RS code 




the number of byte errors for L decisions is a binomial random variable with 

parameters L and P nI .. The mean value of this random variable, is, LP nc ., 
nb oh 

and the standard deviation is /LP (1-P ) . L is sufficiently large for this 

binomial random variable to be well approximated b* a Gaussian random variable 

with the same mean and variance. Since 95.4% of the samples of a Gaussian 

random variable are within the interval specified by the mean plus and minus 

twice the standard deviation, we can be 95% confident that the actual byte- 

error rate for the inner decoder is in the interval P. r +2 /LP (1-P ) . 

oh oh oh 

Such 95% confidence intervals are indicated in Tables II and V. 

The performances of System I for the M = 6, (3,1) inner code and for 
the (18,6) unit-memory inner code are shown in Fig. 7 together with their 
corresponding confidence intervals. We conclude that we may be 95% confident 
that the actual performance of the concatenated coding system deviates no 
more than about 0.1 db from our simulation results. Moreover, since all the 
simulation results are obtained through the same pseudo-random number sequence, 
the relative differences in performance among various systems are, in fact, 
much more accurate than the 0.1 db confidnece interval alone would indicate. 

VIII. SUMMARY AND CONCLUSIONS 

We have extensively studied block-convolutional concatenated coding 
systems with various modifications. We have found that employing unit-memory 
convolutional codes rather than conventional codes can improve the performance 
by nearly 0.3 db. Feedback from the outer decoder to restart a Viterbi 
inner decoder also contributes an improvement of about 0.3 db. But, sur- 
prisingly, feedback from the outer decoder to restart an RTMBEP inner decoder 
provides an approximately 0.5 db advantage; this might be the principal 
occasion where the use of RTMBEP decoding rather than Viterbi decoding is 
justified. Another unexpected result is that soft-decisions by the inner 




decoder in conjunction with an erasures-and-errors outer decoder improves 
the overall performance by only about 0.1 db for RTMBEP decoding. Even with 
Zeoli's modification, which provides an excellent erasure capability, soft- 
decisions in conjunction with an erasures-and-errors outer decoder improves 
performance by about only 0.2 db. 

In Fig. 8, we summarize the effects of each feature discussed above on 
the performance of block-convolutional concatenated coding systems. The 
figure is drawn in terms of a db scale. As a communications engineer starts 
to choose a coding system, the first question he faces is whether his phase- 
locked-loop can tolerate the burden of a rate 1/3 coding system, if the 
answer is positive, he gains 0.5 db over that of a rate 1/2 inner coding 
system. Then, he decides which inner code to employ; to choose the M = 7, 

(3,1) code gives a 0.2 db advantage over the M = 6, (3,1) code but requires 
twice the number of states in the decoder, whereas to choose the M = 1, 

(18,6) code gives a 0.3 db advantage with same number of states, but more 
branch connections required in the inner decoder. The third question is 
whether he will allow the decisions of the outer decoder to be fed back to 
the inner decoder; if not, the obvious choice is Viterbi decoding, otherwise, 
he can gain 0.3 db or 0.5 db depends on whether a Viterbi decoder or an RTMBEP 
decoder is utilized. And finally, if a soft-decision inner decoder is used, 
he can gain 0.2 db through Zeoli's erasure scheme if he uses a Viterbi decoder, 
or gain about 0.05 db if an RTMBEP decoder is employed. 

The leading contenders for a good concatenated system are Zeoli's 
annexation scheme with the unit-memory code (System VI) , or either hard 
decision (System IV) or soft-decision (System V) RTMBEP decoding of the unit- 
memory code with feedback from the outer decoder. Among them, the soft- 
decision RTMBEP decoder with feedback performs the best. In terms of hard- 
ware implementation, Zeoli's modification with the unit-memory code and the 
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hard-decision RTMBEP decoder are of approximately the same complexity. How- 
ever, since the operation of the Viterbi decoder for Zeoli's system depends 
on the correct feedback from the outer decoder, there is always a slim 
chance that the outer decoder fails to provide correct decisions to the 
Viterbi decoder. Since the encoder constraint length is much larger than 
the decoder constraint length, this can cause endless errors as if a cata- 
strophic convolutional code were used. Thus, it is necessary to send 
synchronization signals periodically to reset the Viterbi decoder to guarantee 
restoration of normal operation. The RTMBEP decoder has the same constraint 
length as that of the encoder, therefore the decoder is able to recover 
fro i errors in a few branches by itself without feedback. The feedback from 
the outer decoder only speeds this process up; therefore, when an error is 
fed back, the most damage it can cause is for the RTMBEP decoder to make a 
few more errors before it recovers by itself. This is certainly a very de- 
sirable advantage for a concatenated coding system. Moreover, because the 
decoder can restore its normal operation quickly, the degree of interleaving 
required for this scheme is considerably less than the full Reed-Solomon 
block length interleaving required for the Zeoli's scheme. 

Finally, as a remark to information theorists, we note that for System III 
(theRTMBEP inner decoder for the rate 1/3 (18,6) unit-memory code concatenated 
with the (63,51), 6-error-correcting RS code with feedback from the RS errors- 
only decoder) we can achieve a byte-error-probability of 10 ^ at E^/N^ of 
2.25 db , or, equivalently, at E /N of 3.25 db. The cut-off rate. R , 
of this 8-level quantized AVGN channel is 0.275 whereas its channel capacity 
is 0.44. The overall rate of the concatenated coding system is 0.27. It 
seems that the cut-off rate, rather than the channel capacity, is still the 
practical limit of rate for reliable communications, even for a very 



sophisticated concatenated coding system, just as it is in a conventional 
convolutional coding system employing sequential decoding [16]. The 
advantage of the concatenated coding system resides only in the elimination 
of "deleted data" such as is always present in a sequential decoding system 
because of the latter's highly variable computation. 
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